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The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) fy/IONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S. C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )n Responsive to communication(s) filed on . 

2a)n This action is FINAL. 2b)S This action is non-final. 

3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) ^ C!aim(s) 1-20 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) S Claim{s) 1-20 is/are rejected. 
?)□ C!aim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 
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Replacement drawing sheet(s) including the correction is required if the drawlng(s) is objected to. See 37 CFR 1.121(d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
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DETAILED ACTION 

Claim Objections 

1 . Claims 10, 17 and 20 are objected to under 37 CFR 1 .75(c), as being of improper 
dependent form for failing to further limit the subject matter of a previous claim. 
Applicant is required to cancel the claim(s), or amend the claim(s) to place the claim(s) 
in proper dependent form, or rewrite the claim(s) in independent form. 

The claims are written in the form of a preamble made to depend on another 
claim. The stated preamble is not given patentable weight as it fails to breathe life, 
meaning, and vitality into the claims. As such, the claims fail to further limit the subject 
matter of the claim(s) upon which they depend. See MPEP §§ 608.01 (n) and 2111.02. 

Claim Rejections - 35 (JSC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed 
publication in this or a foreign country, before the invention thereof by the applicant for a patent. 

3. Claims 1-2, 4-7 are rejected under 35 U.S.C. 102(a) as being anticipated by Qian 
et al., US Patent 6721454. 

As in Claims 1 and 10, Qian et al. teaches a method and computer storage 
medium with instructions for obtaining unstructured video frames ("A video sequence 2 
is input", Column 2, lines 64-65), generating segments from the shot boundaries based 
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on the color dissinnilarity between consecutive frames ("A color histogram technique 
may be used to detect the boundaries of the shots", Column 3, lines 42-43), extracting a 
set by processing pairs of segments ("the global motion of the video content is 
estimated 8 for each pair of frames in a shot", Column 3, lines 59-61 ) for their visual 
dissimilarity and temporal relationship, and merging the video segments by applying a 
probabilistic analysis to the extracted set to represent the video structure ("each shot is 
summarized 16 ... events 22 are inferred from the shot summaries by a domain specific 
event inference model". Column 3, lines 6-8). 

As in Claim 2, Qian et al. teaches generating color histograms from the 
consecutive frames and from the histograms, generating a difference signal, 
thresholding of this signal based on a mean dissimilarity over several frames to produce 
a signal representative of the existence of a shot boundary (Column 3, lines 42-50 and 
Figure 5). 

As in Claim 4, Qian et al. teaches morphologically transforming the thresholded 
difference signal with a pair of structuring elements to eliminate the presence of multiple 
adjacent shot boundaries ("When the difference between the histograms of two frames 
exceeds a predefined threshold, the content of the two frames is assumed to be 
sufficiently different", Column 3, lines 45-48). 

As in Claim 5, Qian et al. teaches computing a mean color histogram for each 
segment and a visual dissimilarity feature metric from the difference between mean 
color histograms for pairs of segments (Column 3, lines 42-50 and Figure 5). 
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As in Claim 6, Qian et al. teaches processing pairs of segments for a temporal 
separation between pairs of segments and for an accumulated temporal duration 
between pairs of segments ("each shot is summarized 16 ... events 22 are inferred from 
the shot summaries by a domain specific event inference model". Column 3, lines 6-8). 

As in Claim 7, Qian et al. teaches generating parametric mixture models 
(summaries created by shot summarization 16, Figure 1) to represent class-conditional 
densities of the inter-segment features (based on temporal information and color 
analysis, See Claim 1 rejection supra) that comprise the feature set and applying the 
merging criterion to the parametric mixture models (event inference 20/detected events 
22, Figure 1). 



Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over Qian et 
al., US Patent 6721454. 

Qian et al. teaches obtaining unstructured video frames, generating segments 
from the shot boundaries based on the color dissimilarity between consecutive frames, 
extracting a set by processing pairs of segments for their visual dissimilarity and 
temporal relationship by generating color histograms from the consecutive frames and 
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from the histograms, generating a difference signal, thresholding of this signal based on 
a mean dissimilarity over several frames to produce a signal representative of the 
existence of a shot boundary (See Claim 2 rejection supra) and merging the video 
segments by applying a probabilistic analysis to the extracted set to represent the video 
structure (See Claim 1 rejection supra) and the difference signal to be based on a mean 
dissimilarity over several frames centered on one frame. Qian et al. fails to teach 
basing the number of frames used to calculate the difference signal on a fraction of the 
frame rate of video capture as recited in the claims. Within the field of the invention, it 
would be obvious to one of ordinary skill in the art to base the number of frames on a 
fraction of the frame rate (See also Image Analysis and Mathematic Morphology, Vol. 1 , 
Jean Serra). One would have been motivated to make such a combination because a 
shortened time frame for calculating the difference signal would have been obtained. 

6. Claim 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over Qian et 
al., US Patent 6721454. 

In accordance with Claims 8 and 15, it is notoriously well known that queues are 
used to implement hierarchical displays. The examiner takes official notice of this 
teaching. It would be obvious to one of ordinary skill in the art to combine the use of the 
organizing video segments into hierarchies with a queue implementation. 
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7. Claims 9, 11-15 and 16-20 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Qian et al., US Patent 6721454 and further in view of Qian et al., US 
Patent 6616529. 

As in Claims 9, 1 1. 17-18 and 20, US Patent 6721454 teaches a method and 
computer storage medium with instructions for obtaining unstructured video frames ("A 
video sequence 2 is input", Column 2, lines 64-65), generating segments from the shot 
boundaries based on the color dissimilarity between consecutive frames ("A color 
histogram technique may be used to detect the boundaries of the shots", Column 3, 
lines 42-43), extracting a set by processing pairs of segments ("the global motion of the 
video content is estimated 8 for each pair of frames in a shot", Column 3, lines 59-61) 
for their visual dissimilarity and temporal relationship, merging adjacent video segments 
by applying a probabilistic analysis to the extracted set to represent the video structure 
("each shot is summarized 16 ... events 22 are inferred from the shot summaries by a 
domain specific event inference model". Column 3, lines 6-8), and generating a 
parametric mixture model of the inter-segment features ("In this model inference 
module, a hunt event is inferred after detecting three shots containing hunt candidates". 
Column 1 1 , lines 60-62). While US Patent 6721454 teaches the segmentation due to 
color dissimilarity, extraction due to visual dissimilarity and temporal relationships, 
merging with probabilistic analysis and generation of a parametric mixture model, they 
fail to show the probabilistic analysis to be a Bayesian analysis applied to the 
parametric mixture model, and representing the merging sequence in a hierarchical tree 
structure as recited in the claims. US Patent 6616529 teaches a video segmentation 
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method similar to that of US Patent 6721454. In addition. US Patent 6616529 further 
teaches the probabilistic analysis to be a Bayesian analysis applied to the parametric 
mixture model (Figure 3 and corresponding text in Columns 4-5), and representing the 
merging sequence in a hierarchical tree structure (Figures 2a-2g and corresponding 
text). It would have been obvious to one of ordinary skill in the art, having the teachings 
of US Patent 6721454 and US Patent 6616529 before him at the time the invention was 
made, to modify the segmentation with color dissimilarity and temporal relationships 
with a parametric mixture model taught by US Patent 6721454 to include the 
construction of hierarchy according to probabilistic merging with Bayesian analysis of 
US Patent 6616529, in order to obtain a hierarchical representation of the frames 
grouped by color dissimilarity and temporal relationships according to Bayesian 
probability methods of analysis. One would have been motivated to make such a 
combination because a visual representation of the segmented video would have been 
obtained, as taught by US Patent 6616529 (Column 2, lines 24-55). 

As in Claim 12, US Patent 6721454 teaches computing a mean color histogram 
for each segment and a visual dissimilarity feature metric from the difference between 
mean color histograms for pairs of segments (See Claim 5 rejection supra). 

As in Claim 13, US Patent 6721454 teaches processing pairs of segments for a 
temporal separation between pairs of segments and for an accumulated temporal 
duration between pairs of segments (See Claim 6 rejection supra). 
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As in Claim 14, US Patent 6721454 teaches generating parametric mixture 
models to represent class-conditional densities of the inter-segment features that 
comprise the feature set (See Claim 7 rejection supra). 

As in Claim 15, US Patent 6721454 teaches performing the merging in a 
hierarchical queue by initializing the queue by introducing each feature in the queue 
with a priority of the probability of merging each corresponding pair of segments, 
depleting the queue by merging the segments if the criterion is met, and updating the 
queue based on the updated model (See Claim 8 rejection supra). 

As in Claim 16, US Patent 6721454 teaches representing the merging sequence 
as a hierarchical tree structure (See Claim 9 rejection supra) including a frame extracted 
from each segment and displayed at each node of the tree (Column 10, line 61 - 
Column 11, line 6). 

As in Claim 19, US Patent 6721454 teaches representing the merging sequence 
as a hierarchical tree structure including a frame extracted from each segment and 
displayed at each node of the tree (See Claim 16 rejection supra). 
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Conclusion 



The prior art made of record on form PTO-892 and not relied upon is considered 
pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.1 1 1(c) to 
consider these references fully when responding to this action. The documents cited 
therein teach... [state relationship between the cited patents and the claims, e.g., 
...similar virtual reality simulators for robot kinematic devices and back propagation 
delays in neural networks.] 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Sara M Hanne whose telephone number is (703) 305- 
0703. The examiner can normally be reached on M-F 7:30am-4:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached on (703) 308-31 16. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 




